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ABSTRACT 



Previous research regarding the supervision of 
psychotherapists has been primarily based on the perceptions of supervisors 
and supervisees at various levels of experience. This national survey 
examines the attitudes and beliefs of experts in the field of supervision 
concerning what constitutes effective supervision. A number of themes and 
recommendations were noted in the survey including an emphasis on the 
supervisory relationship, a commitment to supervision, and the adoption of a 
supervisory stance. The results point to the need for clearly articulated 
tasks and procedures; attention to developmental levels; accepting 
responsibility for evaluation and feedback; and developing a clear notion of 
desired outcomes. The opinions expressed in the survey describe the goal of 
effective supervision to be the development of a facilitative supervisory 
relationship characterized by empathy; warmth; trust; mutual respect; and 
flexibility. Appendix includes a copy of the survey. (Contains 4 tables and 
61 references.) ( JDM) 
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Abstract 

Previous research regarding the supervision of psychotherapists has been based primarily 
on the perceptions of supervisors and supervisees at various levels of experience. In this national 
survey, the attitudes and beliefs of supervision “experts” in regards to the provision of effective 
supervision were examined. Results indicated a number of themes and recommendations further 
augmenting the literature related to effective supervision including an emphasis on the 
supervisory relationship, a commitment to supervision and an adoption of a supervisory stance, 
clearly articulated tasks and procedures, attention to developmental level, accepting 
responsibility for evaluation and feedback, and clear notions of the outcomes desired through 
supervision and methods to evaluate these outcomes. 
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What is effective supervision?: A national survey of supervision experts 

The field of psychotherapy supervision has been described as maturing into 
“adolescence” (Bernard & Goodyear, 1998, p. xii) and is now becoming recognized as “the 
critical teaching method” (Holloway, 1992, p. 177) for the development of professional skills 
and identity in the training of counselors and psychologists. Increasingly, supervision standards 
and requirements are being delineated by accrediting bodies and state licensing boards as 
training in supervision skills is becoming a recognized field of study in its own right. This trend 
is reflected in the newly endorsed Model Training Program in Counseling Psychology (Murdock, 
Alcorn, Heesacker, & Stoltenberg, 1998), which identifies “supervision and training” as one of 
ten basic competency areas in the training of counseling psychologists. Perhaps in response to 
recent changes in accreditation criteria in which training in supervision is a required content area 
(American Psychological Association, 1 996), a recent survey of APA approved professional 
psychology programs (Scott, Ingram, Vitanza, & Smith, 1 998), reported that half of the programs 
now either require (30%) or offer as an elective (20%) a didactic course in supervision. A 
similar percentage requires (23%) or offers as an elective (31%) a practicum in supervision. This 
is even more strongly evident in the field of counseling psychology, where 85% of programs 
either offer or require a course in supervision (Scott, et. al., 1998). In an earlier survey (Leddick 
& Stone, 1982), it was found that only 1/3 of counselor preparation programs offered training in 
supervision. Thus, it appears that training in supervision is receiving wider recognition as an 
important element of effective preparation for practice as counselors and psychologists. With 
this increasing acknowledgment of the importance of supervision, it is important for us to 
understand what comprises effective supervision. 

In examining the literature regarding effective supervision two clear themes emerge: 1) 
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the importance of the supervisory relationship in promoting a facilitative learning environment 
and effective outcomes in supervision (Ellis & Ladany, 1997), and 2) the focal tasks that lead to 
counseling competency. In addition, a number of structural factors appear to result in more 
positive perceptions of the supervisory process. 

Good supervisory relationships consist of warmth, acceptance, respect, empathy, 
understanding, trust, and a strong emotional bond (Hutt, Scott, & King, 1983; Ladany, Ellis, & 
Friedlander, 1999; McCarthy, Kulakowski, & Kenfield, 1994; Miller & Oetting, 1966; Shanfield, 
Mohl, Matthews, & Heatherly, 1992; Unger, 1999; Worthen & McNeill, 1996). An effective 
supervisory relationship also encourages thoughtful experimentation and reflection on 
"mistakes" so that they can become learning opportunities (Allen, Szollos, & Williams, 1986; 
Hutt et al., 1983; Nelson, 1978; Ronnestad & Skovholt, 1993; Worthen & McNeill, 1996). In 
summarizing the literature on gender and the supervisory relationship Goodyear and Bernard 
(1998) concluded, “gender likely affects the quality of the supervisory relationship, although the 
particular patterns of these effects are not yet fully understood” (p. 11). When it comes to race 
and ethnicity and the supervisory relationship, the findings are mixed as to whether matching 
supervision dyads for race and ethnicity affects the quality of the relationship. It appears that 
there may be complex factors effecting the relationship rather than simple race or ethnic 
matching (Ellis & Ladany, 1 997). 

Other relationship factors that appear to be important for good supervision include factors 
related to effective evaluation. Lehrman- Waterman & Ladany (2001) found that effective goal 
setting and feedback, the central elements of evaluation, strengthen the supervisory relationship. 
Others have found that supervisee interest and openness to feedback and suggestions contributes 
to satisfaction in supervision (Allen, Szollos, & Williams, 1986; Frame & Stevens-Smith, 1995; 
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Hahn & Molnar, 1991; Hendersen, Cawyer, & Watkins, 1997). Evaluation can also be skewed 
by the nature of the supervisory relationship. Turban, Jones, and Rozelle (1990) found that when 
supervisors liked their trainees they provided more support, more effort and more positive 
evaluations. 

When evaluation is conducted within a supportive relationship, self-disclosure is 
facilitated (Lehrman- Waterman & Ladany, 2001). Supervisor and mutual self-disclosure are 
associated with positive supervision experiences, play a critical role in the supervisory 
relationship, and may be pivotal in creating and maintaining a learning environment at critical 
points in supervision (Black, 1988; Hutt et. al., 1983; Nelson, 1978; Worthen & McNeill, 1996; 
Worthington & Roehlke, 1979). 

The most consistent stated outcome in conducting “good” supervision identified in 
previous studies is helping the trainee develop their counseling skills (Heppner & Roehlke, 1984; 
Kennard et al., 1988; Worthington, 1984; Worthington & Roehlke, 1979). But approaching this 
task may vary depending upon the experience level of the supervisee. Typically, beginners 
desire more structure (Stoltenberg, McNeill, & Crethar, 1994; Tracey, Ellickson, & Sherry, 

1989; Worthington & Roehlke, 1979), although Tracey et. al. (1989) found that urgency of client 
condition was a strong moderating influence on desire for structure. This structure includes 
developing intake skills (Heppner & Roehlke, 1984), didactic training on how to counsel 
(Worthington & Roehlke, 1979), and more time spent on developing self-awareness (Nelson, 
1978). Advanced trainees desire assistance with developing alternative conceptualization skills 
(Heppner & Roehlke, 1984), more emphasis on personal development than technical skills, 
working within a cohesive theory, and clear communication about expectations (Allen et al., 
1986). Interns and more experienced supervisees want to examine more personal issues such as 
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countertransference (Heppner & Roehlke, 1984; Rabinowitz, Heppner, & Roehlke, 1986; Winter 
& Holloway, 1991). Worthington (1984) also found that higher satisfaction ratings were 
correlated with helping trainees develop self-confidence. 

Structural factors also have been found to contribute to positive perceptions of 
supervision. Structural discriminators of quality have included longer duration of training, more 
weekly contact time, more frequent contacts per week, and that good quality supervision 
experiences occurred later in the training sequence (Allen et al., 1986). Nelson (1978) also 
found trainees preferred supervisors who showed interest in supervision, had experience as a 
therapist and were conducting regular therapy, and possessed technical or theoretical knowledge. 
Theoretical similarity has also predicted perceived supervisor effectiveness (Putney, 

Worthington, & McCullough, 1992), suggesting that theoretical influence and matching may 
play some role in satisfaction 

As previous research on effective supervision has been primarily based on the 
perceptions of supervisors and trainees at various levels of experience, the time seems ripe for 
examining the attitudes and beliefs of “experts” in supervision regarding providing effective 
supervision. Through an examination of those well versed in supervision, we can begin to 
explore and establish standards of practice for the field at this stage of understanding. Issues 
regarding supervision effectiveness, outcomes, ethical concerns, evaluation, negative 
experiences, important literature, multicultural counseling competencies, supervision as art or 
science, process versus outcome orientation, and the role of theory are central to understanding 
the current state and practice of supervision. Thus, an examination of the beliefs and practices of 
experts can potentially inform training efforts and encourage further research endeavors. 

Although several surveys have been conducted examining the practice of supervision. 
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such as understanding program practices or prevailing practices of supervisors or supervisees 
(Borders & Leddick, 1988; Freeman & McHenry, 1996; Hess & Hess, 1983; Leddick & Stone, 
1982; McCarthy et al., 1994; Nelson, 1978; Romans, Boswell, Carlozzi, & Ferguson, 1995; 

Scott, et. al., 1998), none have explored the actual beliefs and practices of experts in supervision. 
Consequently, the purpose of the present study was to build upon what the accumulated literature 
suggests regarding aspects of effective supervision from the perspectives of trainees and 
practicing supervisors by examining the beliefs and practices of two sets of experts, literature 
experts and training experts. The assumption is that supervision experts should be implementing 
the best in supervisory practice and that their beliefs and behavior can help inform actual 
practice. 

Method 

Participants 

Supervision experts: Literature. Thirty-four experts from the literature were identified 
and sent surveys. “Expert” was defined as someone who had published at least two articles 
and/or chapters within the past 20 years (1979-1999) on the topic of supervision. The following 
publications were reviewed to identify experts. The Journal of Counseling Psychology. 
Professional Psychology: Research and Practice, Counselor Education and Supervision, and The 
Clinical Supervisor as well as the edited Handbook of Psychotherapy Supervision (Watkins, 
1997) and other supervision books published within the past 20 years. A few authors were 
omitted, since it appeared that their publications might have only focused on training issues in 
general and not specifically on supervision. Eighty-two percent of the experts who responded 
graduated from an APA approved program. The majority of these respondents were from 
counseling psychology programs (76.5%), with clinical psychology representing 1 1.8%, 
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counselor education 5.9% and other 5.9%. These experts had been providing supervision for an 
average of 19.8 years (SD = 6.74), and supervised an average of 147.9 trainees each (SD = 

83.93; range = 50 - 300; Mdn = 100). On these two experience variables, the literature experts 
had significantly more experience than the training experts: years providing supervision F (1, 
255) = 4.44, g < .041; number of trainees supervised F (1, 104952) = 38.79, g < .000. The 
majority had taken a class in supervision (56.3%; N = 9), while 68.8 % (N = 1 1) stated that they 
had taught a class in supervision. 

Supervision experts: Training Directors. A list of training directors was obtained from 
the Association of Counseling Center Training Agencies (ACCTA) (N = 128). Seventy-eight 
percent graduated from an APA approved program. The majority of these respondents were 
from counseling psychology programs (58.3%; N = 21), with clinical psychology representing 
13.9% (N = 5), unknown 1 9.4% (N = 7) and other 8.3% (N = 3). These experts had been 
providing supervision for an average of 13.96 years (SD = 7.83), and supervised an average of 
41.6 trainees each (SD = 26.42; range = 13 - 125; Mdn = 35.0). Fifty-four percent (N = 19) 
reported that they had taken a class in supervision, while 22 . 9 % (N = 8) stated that they had 
taught a class in supervision. 

Procedure 

A questionnaire and return envelope were sent to both the literature experts and the 
training experts late in the spring of 1999. A follow-up questionnaire was sent the following fall. 
The overall return rate was thirty-three percent (N = 53). Literature experts had a 50% return 
rate (N = 17; 6 women, 1 1 men; 16 European- Americans, 1 Hispanic), while training experts had 
a 28.0% return rate (N = 36; 21 women, 14 men, 1 blank; 33 European- Americans, 1 Asian- 
American, 2 Hispanic). 
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Measure 

The Supervision Beliefs and Practices Survey (See Appendix) was created for this study 
by including a global survey of supervision issues. The format of the questionnaire included a 
variety of response formats e.g., open-ended questions, anchored items, listing items, etc. The 
survey item domains covered issues regarding supervision effectiveness, outcomes, ethical 
concerns, evaluation, negative experiences, important literature, multicultural counseling 
competencies, supervision as art or science, process versus outcome orientation, and the role of 
theory. This format was used to elicit information about beliefs and practices of experts from an 
exploratory rather than a confirmatory perspective. Demographic items gathered included: 
gender, race, field of study, years providing supervision, how many people supervised, 
supervision experiences, etc. 

Results 

Data analysis followed an inductive content analysis procedure similar to the one outlined 
by Patton (1990), where categories, patterns, and themes were identified from the data for all 
open-ended questions. Simple descriptive statistics were also used, as well as ANOVAs for 
semantic differential items, experience variables, and estimations on positive, negative, and 
neutral supervision experiences. 

Recommendations for providing effective supervision 

This content area was obtained by asking for the “three most important recommendations 
for effective supervision.” Six general themes and eleven subthemes were identified from a total 
of 152 responses (Table 1). Responses grouped under the theme labeled “Procedural” were over 
twice as likely to be noted by training experts than literature experts. Both percentages of 
themes and subthemes are listed to give an indication of the frequency items were mentioned. 
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Common problems leading to negative supervision experiences 

These themes were generated from the responses to the question asking about the “three 
most common problems leading to negative supervision experiences.” Six themes and 13 
subthemes were identified from a total of 151 responses (Table 2). Training experts identified 
items relating to a “Neglecting Authority Role” theme almost three times more often than 
literature experts. This was also true of the subtheme “lack of training” under the theme 
“Supervisor Issues.” Four literature experts noted impairment issues under the “Supervisor 
Issues” theme, while only one training expert stated this as a problem. 

Recommended readings for supervisors 

Table 3 lists the top ten recommended readings for supervisors, which included those 
readings endorsed three or more times. Along with these top ten, 36 other specific readings 
were mentioned along with several general resources (e.g., ethical guidelines, multicultural 
readings, supervision journals, etc.). It might be noted that the most recommended reading 
received 20 endorsements, which represented thirty-eight percent of the respondents. Only three 
experts, representing six percent in each case, identified five out of the top ten, suggesting either 
unfamiliarity with the supervision literature or a wide divergence in the literature of supervision 
considered important. 

Factors influencing development as a supervisor 

This item was generated by having experts list the two most important influences in their 
development as a supervisor. Three general clusters influencing development as a supervisor 
were identified from a total of 107 responses: Supervisors - good and bad (36.4%), 
course/readings (25.2%), and mentors/colleagues (20.6%). An “other” category included 17 
responses (15.9%) and consisted of items such as “experiencing own therapy,” “providing 
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supervision,” and “conducting research on supervision.” 

Supervision topics that need further investigation 

Experts were asked about supervision topics we know too little about. As might be 
expected there was a broad range of responses. Nine categories were identified as well as an 
“other” category, based on a total of 1 16 responses. The nine categories consisted of the 
following from most to least represented: supervision outcomes (19.8%) (e.g., the effect of 
supervision on client outcomes, what constitutes successful supervision), multicultural and 
diversity issues (19.4%) (e.g., effects of culture, race, gender, sexual orientation, etc.), process 
issues (13.8%) (e.g., facilitating reflection, dealing with termination issues, dealing with anger in 
supervision, etc.), ethics (12.1%) (e.g., supervision and therapy boundary issues, multiple roles, 
sexual attraction, etc.), measuring and evaluating supervision (7.8%) (e.g., how to measure 
progress, how to evaluate, supervisor resistance to evaluating supervisees, etc.), developmental 
issues (6.0%) (e.g., supervisor development, development of post Ph.D. professionals, etc.), 
matching issues (6.0%) (e.g., theoretical differences, best dyads for learning, etc.), relationship 
factors (6.0%) (e.g., effects of ruptures in alliance, transference and countertransference in 
supervision, etc.), and supervision models (4.3%) (e.g., models of group supervision, “whys” of 
the theory, etc.). The other category (6.9%) consisted of items such as “supervision of 
supervision,” “limitations of treatment options,” and “supervisee ratings of most important 
supervision activities.” Four respondents did not complete this item. 

Common ethical issues addressed in supervision 

There were 77 total responses to the item asking about the most common ethical issues 
addressed in supervision. Eight categories were identified along with an “other” category. Three 
respondents left this item blank. The eight categories from most to least included: client welfare 
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and supervisee competency (19.5%) (e.g., good client care when supervisee is struggling, relying 
on self-report data, ability to handle client problems, etc.), boundary issues (18.2%) (e.g., 
maintaining effective personal boundaries, how much self-disclosure is appropriate, 
understanding the differences between therapy and therapeutic supervision, etc.), confidentiality 
and informed consent (16.9%) (e.g., when to breach confidentiality, release of information, 
informed consent about treatment, etc.), mandated reporting (10.4%) (e.g., reporting abuse, duty 
to warn, etc.), multiple roles (10.4%), liability concerns (9.1%) (e.g., poor record keeping, issues 
around taping sessions, etc.), high risk situations (7.8%) (e.g., suicidal clients, hospitalization, 
danger to others, etc.), and follow-up (3.8%) (e.g., making sure referrals take, abandonment 
issues, attention to follow-up). The other category consisted of three responses regarding how to 
teach/model corrective learning, supervisee motivations for career, and bad supervisors. 
Activities to increase multicultural counseling competencies 

The question regarding efforts used to increase multicultural counseling competencies 
yielded 102 responses that were divided into 5 categories and an “other” category. These are 
listed from most too least used: didactic education (33.3%) (i.e., readings, seminars, labs, and 
inservices), discussion of client characteristics in supervision (32.4%) (e.g., culture, ethnicity, 
gender, sexual orientation, etc.), exploring personal experiences with diversity (19.6%) (e.g., 
self-exploration, personal therapy, evaluating assumptions, etc.), insuring a diverse counseling 
caseload (5.9%), and providing diverse supervisors or consultants (5.9%). The “other” category 
consisted of three responses, “develop multicultural guidelines for supervision,” “see people as 
people,” and “I don’t do this well.” Two respondents did not complete this item. 

Supervision as art or science 

Two questions comprised this issue, one asking about their belief regarding whether 
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supervision as currently practiced was more art or science and a second question asking whether 
supervision should ideally be practiced as more art or science. These questions were designed to 
gain a sense of the role of science in informing supervision. The question was ordered on a ten- 
point scale from art on one end to science on the other. There was no difference between 
literature and training experts on the current state of supervision (M = 4.2; SD = 1 .50; N = 5 1). 
There was a significant difference between the experts on the ideal state of supervision (ANOVA 
= .046), with a mean of 6.3 (SD = 1 .68) for literature experts and 5.3 (SD = 1 .37) for training 
experts, suggesting that the literature experts believe the practice of supervision should be more 
informed and guided by science than training experts, although both seemed to agree that some 
balance is necessary. The difference between belief in current practice and ideal practice was 
also significantly different for the combined groups (t = .000), with a total mean of 4.2 for the 
current practice and 5.6 for ideal practice, indicating that for both groups, supervision should be 
practiced with more of an informed scientific approach than is currently in place. 

To what degree is supervision guided by theory 

This question inquired as to how much experts were guided by theory in providing 
supervision, again using a 10 point scale with an anchor of “none” on one end and “completely” 
on the other. Experts also differed on this item (ANOVA = .002), with literature experts being 
guided more by theory than training experts (literature experts; M = 8.0; SD =1.17; training 
experts; M = 6.4; SD = 1.85). A follow-up question inquired “if they were guided by a theory, 
which theory did they prefer.” A total of fifty responses were obtained, with some form of 
developmental model endorsed by 62 % of the respondents, object relations/psychodynamic by 
12%, and eclectic/composite models by 6%. The Discrimination model, working alliance model, 
and Kell and Mueller’s ideas each received endorsements by 4%, and there were four other 
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models/theories mentioned by 8% of respondents. Three respondents did not answer this item. 
Negative supervision experiences 

Experts were asked whether they had ever had a negative supervision experience and if 
so, how many. There was no difference between groups on this item. The combined experts 
reported 76.9% had at least one negative experience, with the mean number of negative 
experiences being 1 .9 (SD = 1 .5). Because of the way the question was stated it cannot be 
determined whether these experiences were from the same supervisor or from multiple 
supervisors. 

Estimated occurrence of positive, negative, and neutral supervision experiences 

In a related question to the one regarding negative supervision experiences, experts were 
asked to estimate to what extent they believed supervision experiences were positive, negative, 
or neutral. There were no differences between experts. They indicated that they believed 61 .9% 
(SD = 18.70; range = 25 - 95%) of experiences were positive, 16.1% (SD = 10.51; range = 0 - 
50%) were negative, and 21.4 (SD = 13.19; range = 0 - 60) were neutral. It should be noted that 
there was a wide range of responses to these items. 

Should supervision focus more on process or outcome 

This question was investigated by using a 10 point scale with “process” as an anchor on 
one end and “outcome” on the other. There was no difference between the experts. The experts 
had a mean of 4.7 (SD = 1 .44) indicating a slight orientation towards focusing on process versus 
outcome. 

Outcomes of effective supervision 

This content area was obtained by having experts list the 3 most important outcomes 
expected from effective supervision. Four themes and seven subthemes were identified based on 
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150 responses (Table 4). Training experts were twice as likely to generate items related to the 
theme “Increased self-awareness.” 

Evaluating effective supervision 

This item was explored by asking about the most effective methods for evaluating 
supervision as well as recommended procedures and instruments. The question regarding 
methods of evaluation produced 75 responses that were divided into 5 categories and an “other” 
category as well as 5 responses that stated either they did not know of any effective methods or 
were unclear how to evaluate effectively. These categories are listed from most too least used: 
objective measures (24.0%) (e.g., measuring goal attainment with specific measures, use of the 
OMART, ratings, etc.), self reports/self evaluations (21.3%) (e.g., supervisee and supervisor 
satisfaction, reports of effectiveness, reactions of supervisees, etc.), supervision 
interviews/feedback (14.7%) (e.g., ongoing feedback, meetings to discuss supervisory 
relationship, issues discussed in an interview, etc.), evaluating client outcome (12.0%) (e.g., 
client ratings of helpfulness, client improvement ratings, outcome data, etc.), evaluating 
video/audio taped sessions (12.0%), and “other” (9.3%) (i.e., formal case presentation, 
examining confidence, factors measured rather than methods). 

Instruments or procedures used in evaluation generated 59 responses. Instruments 
comprised 49.2% of the responses and were divided into two categories: in-house instruments 
(37.3%) and formal instruments (1 1.9%, Role Conflict and Role Ambiguity Inventory, 
Supervisory Styles Inventory, Working Alliance Inventory, Supervision Questionnaire, Swain 
and Hess Rating Scale, Aldrich and Hess Rating Scale). Literature experts mentioned all of the 
formal instruments. There were ten different procedures listed which comprised 27.1% of the 
responses. The procedures with multiple responses consisted of: reviewing videotapes (6.8%), 
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ongoing discussion/feedback (5.1%), and assessing client outcomes (3.4%). The following were 
mentioned only once: conducting time series measures to determine changes, assessing critical 
incidents, narrative reports, peer review, semi-structured interviews, single subject intensive 
observations, and assessing accomplishment of supervision plans. 

Discussion 

The beliefs and practices of experts can be used as a measuring stick for a standard of 
practice for supervision at this point. From this study, the two predominant recommendations for 
providing effective supervision encompassing 49% of responses were non-technical in nature; 
developing an effective supervisory relationship, and an attitude and approach to supervision that 
takes this enterprise seriously. These findings are similar to previous studies (Allen et al 1986; 
Shanfield et al, 1992) and continue to validate the importance of the supervisory relationship. 

The other responses referred to the specific tasks, procedures or theoretical approaches and 
understanding of the supervisor role. 

In regards to factors that contribute to negative supervision experiences, over one-third of 
the responses indicated that issues deriving from the supervisor are primary. This was endorsed 
twice as much as the second most prevalent theme, supervisee issues. Three other themes also 
pointed towards problems related to the supervisor including mismatches, unsafe environments, 
and neglecting the authority role. Negative experiences arising from supervisee characteristics 
accounted for about one-fifth of responses. Thus, it appears that much of the remediation or 
prevention of negative supervisory experiences should be directed towards more effective 
training of supervisors. 

In relation to the question of training, experts state that the most important influences in 
their development as supervisors are other supervisors, which has been found previously (Guest 




17 



Effective Supervision 1 7 



& Beutler, 1988), courses and readings, and mentors/colleagues. Since supervisors are a critical 
influence on the development of future supervisors, and since they also seem to be the source of 
most supervision difficulties, efforts to monitor and evaluate the effectiveness of supervision 
should be instituted to insure good quality supervision. 

Although three-fourths of experts experienced at least one negative experience in their 
supervision, it would be inappropriate to conclude that every negative experience is counter- 
productive. At times, negative supervision events are powerful learning experiences with 
positive outcomes. Although we acknowledge that negative experiences in supervision can 
produce positive outcomes, generally negative experiences hamper the learning environment and 
damage the supervisory relationship (Gray, Ladany, & Walker, 1999) and should be minimized 
and eliminated as much as possible. In this regard, experts estimate that positive experiences 
occur approximately 62% of the time while negative experiences occur 21% of the time. Thus, it 
appears that more research and training is warranted in an effort to facilitate more positive and 
effective supervision. 

Experts are generally guided by theory in providing supervision and the stated theory of 
choice for many at this point is developmental. Further evidence for the influence of 
developmental models in supervision comes from experts citing Stoltenberg et. als (1997) book 
on developmental models as the most endorsed reading for supervisors. This may be somewhat 
controversial since it appears that there is still disagreement at this point regarding whether 
empirical support is available to support developmental models. Ellis and Ladany (1997) in their 
critical review of the literature on research related to developmental models state that much of 
the research so far has been flawed and therefore suspect. Others have strongly argued that 
research support is mounting to support tenets of developmental models (Stoltenberg, McNeill, 
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& Crethar, 1994; Watkins, 1995b; Worthington, 1987). 

As far as desired outcomes, experts suggest that professional development and supervisee 
skill improvements are the most important outcomes. It is of interest to note that positive client 
outcomes were the third most important outcome at 16%. Yet, it appears that one of the most 
important reasons for professional development and improved skill would be to improve client 
outcomes (Holloway & Neufeldt, 1995; Lambert & Ogles, 1997; Stein & Lambert, 1995). 
Holloway and Neufeldt (1995) made this critical statement in regards to supervision outcome “it 
is disconcerting that supervisors, who have the responsibility to insure the therapist’s competent 
practice with clients, are perhaps more influenced by the trainee’s interpersonal involvement in 
supervision than their effectiveness with the client” (p. 21 1). Lambert and Hawkins (2001) 
stated that client outcomes are “one of the most meaningful tests of the efficacy of clinical 
supervision” (p. 131). In light of these statements about supervision outcomes, we have yet to 
answer definitively whether the development of specific therapy skills actually affects client 
outcome. In fact, attempts to standardize skills and treatment protocols through manuals suggest 
some potential harm to client outcomes, such as decreased relationships skills and alliance 
building, less support, approval, optimism, and increased defensiveness and authoritative 
behaviors (Henry, Schacht, Strupp, Butler, & Binder, 1993; Henry, Strupp, Butler, Schacht, & 
Binder, 1993; Rounsaville, Chevron, Weissman, 1984). After reviewing a number of specific 
therapy models and manualized treatments, Wampold, Ahn, and Coleman (2001), citing 
Wampold (2001) concluded “It appears that adherence to a manual (or even use of a manual) is 
unrelated to [therapy] outcome” (p. 269). Thus, it is yet unclear how skill development and 
outcomes are related. Ladany, Ellis, & Eriedlander (1999) suggested that common factors “may 
play a more significant role in the outcome of supervision than any specific approaches or 
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techniques” (p. 447). These common factors may include such things as the supervisory working 
alliance. 

When experts were asked what areas need further investigation, understanding desired 
supervision outcomes received the most endorsements. Additionally, experts indicated that a 
balance should be maintained in supervision between focusing on process versus outcome. As 
stated earlier, little has been done to date on how to address and measure outcomes as related to 
supervision. Thus, as noted by Holloway and Neufeldt (1995) it appears that the role of 
supervision outcomes needs further clarification, and that measuring those outcomes is still 
problematic. For example, the following research questions may take us in that direction: 

“What is the relationship between desired and expected skill development in supervisees and 
client outcomes?” And “What skills are related to therapy outcomes?” New procedures that 
incorporate client outcome data into supervision, such as the one proposed by Lambert and 
Hawkins (2001) may prove to have a significant impact on the practice of supervision and lead to 
outcome driven supervisory practices, without sacrificing attention to supervision and therapy 
processes. The supervision outcomes we focus on will influence the purposes, goals, and 
operating procedures of supervision and therefore should be clearly articulated. 

The second most frequently indicated area for further investigation was that of 
multicultural and diversity counseling competence. This has received increased attention in 
recent supervision research (Aponte & Johnson, 2000; Helms & Cook, 1999; McNeill, Horn, & 
Perez, 1995). It appears that the most common methods to strengthen these competencies are 
direct education, discussion of these issues in supervision, and self-exploration. Perhaps more 
use can be made of multicultural and diverse supervisors and consultants as well. 

This study identified the most commonly occurring ethical issues in supervision as 
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experienced by experts. This study found that those issues related to client welfare were rated as 
the most prevalent concerns. However, it did not examine frequency or impact on supervision. 
In a recent study, Ladany, Lehrman- Waterman, Molinaro, and Wolgast (1999), found that “47% 
of the variance in the perceived supervisory working alliance can be accounted for by 
supervisors’ ethical adherence to ethical practice in supervision guidelines” (p. 466). This 
suggests that attention to ethical practice in supervision and modeling of ethical behavior can 
have a significant impact on the quality of supervision experienced. 

In regards to evaluating effective supervision, it appears that the experts struggle with the 
task of evaluation. The literature experts mentioned a few formal instruments, but the training 
experts cited none. Most instruments used were of the in-house variety and seemed to vary from 
satisfaction measures to more specific measures of particular skills. Evaluation procedures 
included observing tapes, giving feedback, and assessing client outcomes. These findings 
suggest that there is a genuine need to develop refined instruments and procedures along with 
policies that encourage and establish as standard practice the need for evaluation. In regards to 
this finding, one of the practices identified by experts leading to negative supervision 
experiences was “neglecting the authority role,” which included avoiding evaluation. More 
attention to and training in the task of evaluation in supervision is clearly warranted. This is 
especially critical given the findings of Lehrman- Waterman and Ladany (2001) that indicate that 
effective evaluation procedures can strengthen the supervisory relationship, which seems to 
enhance supervisory outcomes. Previous studies have also shown that effective evaluation can 
lead to good supervisory experiences (Allen et al, 1986; Henderson, Cawyer, & Watkins, 1999). 

In summary, effective supervision according to the experts consists of a good working 
alliance, an attitude of serious commitment to supervision, science informed practice, clearly 
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articulated tasks and procedures, and attention to developmental level. There also needs to be 
more attention to the continued training of effective supervisors, developing multicultural 
counseling competence, clarity in regard to desired outcomes, effective evaluation instruments 
and methods to evaluate those outcomes, and a commitment to informed ethical practice. 

The results of the present study are consistent with previous investigations and 
conceptualizations based on supervisors’ and supervisees’ perspectives that emphasized the 
central role of the supervisory relationship (e.g., Shanfield et al., 1992; Worthen & McNeill, 
1996; Watkins, 1995a), attention to supervisory tasks (e.g., Kennard et al., 1998), and structural 
variables (e. g., Kenfield & McCarthy, 1994) in effective supervision. It points to the need for 
further investigation of evaluation methods, attention to client outcome in supervision, 
multicultural supervision procedures, and more effective training of supervisors. 

It appears that a primary goal of effective supervision should be the development of a 
facilitative supervisory relationship characterized by empathy, warmth, trust, mutual respect, and 
flexibility. This goal is supported by an investigation by Shanfield, Mohl, Matthews, and 
Hetherly (1992) in which supervisor empathy accounted for 72% of the variance in rater- 
perceived excellence of supervisors. While this goal may appear self-evident and has been 
echoed by other researchers, previous research (e.g., Galante, 1988), as well as the current study 
of experts continues to document a high percentage of negative supervision events. 

Consequently, we believe that more emphasis should be placed on the supervisor-supervisee 
relationship including a study of the critical factors in creating healthy relationships, 
developmental influences, and the learning outcomes of such relationships. As initially 
suggested by Loganbill, Hardy, and Delworth (1982), perhaps the supervisory relationship is 
essential in supervision analogous to the importance of the psychotherapeutic relationship as a 
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necessary base for all later interventions. 

Our results are subject to the usual limitations of survey methodology including sample 
ranges and size. The length and design (i.e., many open-ended questions) of the survey likely 
played a role in a lower return rate. However, our a priori decision was that the tradeoff in return 
rate would be worth the type of rich qualitative data that would be generated in this broad global 
survey. As such, the results yielded by the present study of experts further adds to the growing 
literature on the training and supervision of psychotherapists and counselors in pointing to future 
directions for theory and research into supervision processes and outcomes. 
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Table 1 

Recommendations for Effective Supervision 



Content Themes and 
Sub Themes 


Definition 


Examples 


% 

Responses 
(N= 152) 


Attend to the 


Creating a supervisory 


Holding environment, core 


28.3% 


Relationship 


relationship that 


facilitative conditions, 




- Safe and 


creates an optimal 


trusting and respectful 




nonthreatening working 
alliance (15.13%) 

- Balance of challenge 
and support (13.1 6%) 


learning environment 


relationship 




Supervision Stance 


Attitude towards 


Come prepared, take 


21.1% 


- Openness (5.92%) 


supervision, 


seriously, acknowledge 




- Take seriously 
(3.95%) 

- Inspire (2.63%) 

- Other (8.55%) 


supervisee and self 


limitations, 
nondefensiveness, 
modeling effective 
behavior, humility 
regarding the growth 
process, encourage wonder 




Tasks 


The focal activities of 


Keep client welfare in 


19.1% 


- Provide positive and 


supervision designed 


mind, help supervisee 




constructive feedback 


to facilitate desired 


develop own orientation, 
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(6.58%) 


outcomes 


direct observation, 




- Needs assessment 




understand supervisee 




(5.92%) 




needs, provide feedback 




- Observe work (3.95%) 








- Other (2.63%) 








Procedural* 


Establishing the 


Establish goals early, 


13.8% 


- Establish objectives 


parameters, 


communicate clear ground 




(7.24%) 


objectives, 


rules, attend to 




- Clarify expectations 


expectations, and 


professional development 




(3.29%) 


procedures of 


as well as clinical skills, 




- Discuss theoretical 


supervision 


set appropriate boundaries 




approach (2.63%) 








Think Developmentally 


View supervision and 


Tailor the supervision to 


9.9% 




supervisees as having 


the level of the supervisee, 






different needs 


recognize developmental 






depending on level of 


stages, think in terms of 






experience and 


development 






conceptual ability 






Supervision Perspective 


Approaching 


Think like a supervisor, 


7.9% 




supervision as a 


know the supervision 






distinct learning 


literature, decide on a 






endeavor 


model to follow, good 





clinician does not 
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necessarily make a good 
supervisor, be aware of 
parallel process 



* = Training experts were over twice as likely to note these items than literature experts. 
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Table 2 



Common Problems Leading to Negative Supervision Experiences 



Content Themes and 
Sub Themes 


Definition 


Examples 


% 

Responses 
(N = 151) 


Supervisor Issues 


The way the 


Over controlling, 


38.4% 


- Style (20.53%) 


supervisor operates 


dogmatic, lack of training, 




- Don’t invest (5.96%) 


leads to negative 


not following a model, not 




- Lack of training 
(5.30%)* 

- Impairment (3.3 1%) + 


experiences 


committed, too busy, 
unresolved issues, abuse of 
power 




Supervisee Issues 


Supervisee 


Inability or unwillingness 


19.2% 


- Non learning attitude 


characteristics lead to 


to receive feedback, 




(7.95%) 

- Style (5.30%) 

- Impaired (3.31%) 


negative experiences 


resistant to new learning, 
lack of patience, 
argumentative, rigid 
adherence to a certain 
theory, unresolved issues, 
easily offended 




Mismatch 


Conflicts that arise 


Personality conflicts that 


14.6% 


- Personality/style 


from a difference in 


are not dealt with, 




(7.95%) 


approach and do not 


interpersonal difficulties, 




- Theoretical (3.97%) 


come from 


clash of world views, 





O 

ERIC 
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impairment, yet are 


differing theoretical 






not transcended 


orientations 




Unsafe Environment 


Environment 


Overly critical, no positive 


11.9% 


- Too critical (3.31%) 


experienced as 


working alliance, not safe 




- Supervision and 


threatening, disrupting 


to reveal, power struggles. 




therapy blurring 


learning 


supervision therapy lines 




(2.65%) 




crossed 




Neglecting Authority 


Avoiding or 


Not discussing evaluation. 


11.9% 


Role 


minimizing the need 


fears of evaluation not 




- Unclear expectations 


for some type of 


discussed, not setting 




(6.62%)* 


structure and 


expectation for evaluation. 




- Avoiding evaluation 


evaluation 


role expectations unclear. 




(5.30%)* 




tacit assumptions, no 








objectives or agenda 




Environmental 


The setting or context 


Not an agency priority. 


3.3% 


Influences 


leads to negative 


institutional interference. 






experiences 


lack of organization in how 








assigned and performed 





* = Training experts identified these items almost three times more frequently than literature 
experts. 



+ = Only one training expert identified, while four literature experts identified these items. 
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Table 3 

Ten Most Recommended Supervision Readings 

Recommended Readings 



Stoltenberg, C. D., McNeill, B. W., & Delworth, U. (1997). IDM: 

An integrated developmental model for supervising counselors and 
therapists. San Francisco: Jossey-Bass Publishers. 

Bernard, J. M., & Goodyear R. K. (1998). Fundamentals of clinical 
supervision (2 nd . ed). Boston: Ally and Bacon. 

Loganbill, C., Hardy, E., & Delworth U. (1982). Supervision: A 
conceptual model. Counseling Psychologist, 10(1) . 

Watkins, C. E., Jr. (Ed.) (1997). Handbook of psychotherapy 
supervision. New York: Wiley. 

Skovholt, T. M., & Ronnestad, M. H. (1). The evolving professional 
self: Stages and themes in therapist and counselor development. New York: 
Wiley. 



Percent 
Endorsing 
37.7 (N = 20) 



28.3 (N = 15) 
15.1 (N = 8) 



9.4 (N = 5) 



7.6 (N = 4) 



Hess, A. H. (Ed.) (1980). Psychotherapy supervision: Theory, 5.7 (N = 3) 

research, and practice. New York: Wiley. 

Holloway, E. L. (1995). Clinical supervision: A systems approach. 5.7 (N = 3) 
Thousand Oaks, CA: Sage. 

Pope-Davis, D. B., & Coleman, H. L. (Eds.) (1997). Multicultural 5.7 (N = 3) 
counseling competencies: Assessment, education and training, and 
supervision. Thousand Oaks, CA: Sage. 
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Russell, R. K., Crimmings, A. M., & Lent, R. W. (1984). Counselor 
training and supervision: Theory and research. In S. D. Brown & R. W. 

Lent (Eds.), Handbook of counseling psychology (pp. 625-681). New York: 
Wiley. 

Loganbill, C., Hardy, E., & Delworth U. (Eds.). (1982). Supervision 
in counseling I [Special issue]. Counseling Psychologist 10(1) . 




5.7 (N = 3) 



5.7 (N = 3) 
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Table 4 

Most important outcomes of effective supervision 



Content Themes and Definition Examples % 

Sub Themes Responses 

(N = 150) 



Professional 


Developing 


A sense of self-efficacy. 


32.7% 


development 


confidence in their 


confidence, enhanced 




- confidence (10.67%) 


professional role. 


autonomy, growth as a 




- growth (8.67%) 


while maintaining an 


clinician and professional. 




- professional 


orientation towards 


faith in the profession. 




identity/ generativity 


continuous learning 


motivated for a career in 




(8.67%) 




psychology, healthy 




- continuous learning 




respect for their power. 




orientation (4.67%) 




openness to new ideas 




Supervisee’s skills 


Cognitive and/or 


Enhanced skills. 


26.7% 


improve (competence) 


behavioral skills are 


conceptualizes more 






improved leading to 


effectively, improved 






more effective 


ability to work, better 






interventions 


insight, increased clinical 








judgement, improved 








relationship skills 




Positive client outcomes 


The client is impacted 


Client improves, client 


16.0% 




in a positive manner 


learns something, client is 
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Increased self- 
awareness* 

- self supervising 
(4.67%) 

- self as therapeutic tool 
(4.67%) 

- knowing self (2.67%) 
Other 






satisfied, client well-being 
is safe-guarded, reduction 
in symptoms, clients keep 
appointments 

Focus on self- Ability to reflect and track 14.7% 

monitoring, own work, know when 

knowledge of over head and refer, 

motives, intents, and understanding “blind 

self as part of the spots” and 

system and learning to countertransference, 

use self as a learning more about self 

therapeutic tool 

Supervisee satisfaction, 9.3% 

depends on stage of 

development, become 

more process oriented, 

boundaries maintained, 

accomplishing training 

goals 



= Training experts were twice as likely to generate these items than literature experts. 
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Appendix 

Survey of Supervision Beliefs and Practices 



You have been identified as an expert in supervision and this survey is designed to elicit your 
thoughts and beliefs regarding the practice of psychotherapy/counseling supervision. 

1 . Identify the three most important recommendations you would make to supervisors about 
providing effective supervision? 

2. What would you say are the 3 most common problems that lead to negative supervision 
experiences? How often do you think they occur by percentage of supervision dyads? 

3. What are the three (or more) most important readings (i.e., books, articles, chapters, 
papers, etc.) you would recommend to supervisors? 

4. What were the two most important influences (e.g., courses, books, supervisors, mentors, 
etc.) in your development as a supervisor? 

5. What are three areas/topics of supervision that we know too little about? 

6. What are the two most common ethical issues that you find yourself addressing in the 
supervision you provide? 

7. What are two things that you do to increase multicultural counseling competencies in the 
supervisees that you supervise? 

8. Is supervision as currently practiced more of an art or science? (Circle the number that 
best represents your opinion) 

Art 12345678910 Science 

9. Should supervision ideally be practiced more as an art or science? (Circle the number 
that best represents your opinion) 

Art 123456789 10 Science 

10. To what degree are you guided by a theory/model in providing supervision? 

None 12 345 6 78910 Completely 

If you are guided by theory/model, what theory guides your supervision? 

1 1 . What theory or model of supervision might be recommended to guide the beginning 
supervisor? 
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Have you ever had a negative experience as a supervisee? Y N 
If yes, approximately how many negative experiences? 

What percent of supervision experiences, overall, do you think are positive, negative, 
or neutral? 

Positive % 

Negative % 

Neutral % 

Is it more important to focus on therapeutic process (i.e., conceptualization, relationship 
issues, etc.) or therapeutic outcomes in supervision (i.e., symptom reduction, is the client 
reporting improvement, etc.)? 

Process 12345 67 8910 Outcome 

What would you say are the three most important outcomes of effective supervision? 
What is the most effective way to evaluate the effectiveness of supervision? 

What procedures or instruments would you recommend for evaluating supervision? 



Sex: □ Female □ Male 

Minority Status: □ African American □ Asian Heritage □ Hispanic 

□ Native American □ Pacific Islander □ Other 

□ Multi-Ethnic □ European-American 

Did you graduate from an APA approved program? □ Y □ N If yes, check one: 
Clinical Psychology Counseling Psychology Other 

How many years have you been providing supervision? 

Approximately how many supervisees have you supervised? 

Have you had a class in supervision? Y N 

Have you taught a class in supervision? Y N 
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